On estimating the probability multiset

Many statistical properties are determined not by the probabilities of the possible outcomes, but just by the multiset of these probabilities. We estimate this probability multiset from an observed data sample. Akin to standard maximum likelihood, which maximizes the probability of the observed outcomes, we propose profile maximum likelihood (PML), which maximizes the probability of the observed profile—the number of symbols appearing any given number of times. We prove the existence and consistency of the PML estimate, establish some of its modeling and functional properties, and derive it in closed form for several simple and short profiles.